Image manipulation localization aims at distinguishing forged regions from the whole test image. Although many outstanding prior arts have been proposed for this task, there are still two issues that need to be further studied: 1) how to fuse diverse types of features with forgery clues; 2) how to progressively integrate multistage features for better localization performance. In this paper, we propose a tripartite progressive integration network (TriPINet) for end-to-end image manipulation localization. First, we extract both visual perception information, e.g., RGB input images, and visual imperceptible features, e.g., frequency and noise traces for forensic feature learning. Second, we develop a guided cross-modality dual-attention (gCMDA) module to fuse different types of forged clues. Third, we design a set of progressive integration squeeze-and-excitation (PI-SE) modules to improve localization performance by appropriately incorporating multiscale features in the decoder. Extensive experiments are conducted to compare our method with state-of-the-art image forensics approaches. The proposed TriPINet obtains competitive results on several benchmark datasets.

We propose the first joint audio-video generation framework that brings engaging watching and listening experiences simultaneously, towards high-quality realistic videos. To generate joint audio-video pairs, we propose a novel Multi-Modal Diffusion model (i.e., MM-Diffusion), with two-coupled denoising autoencoders. In contrast to existing single-modal diffusion models, MM-Diffusion consists of a sequential multi-modal U-Net for a joint denoising process by design. Two subnets for audio and video learn to gradually generate aligned audio-video pairs from Gaussian noises. To ensure semantic consistency across modalities, we propose a novel random-shift based attention block bridging over the two subnets, which enables efficient cross-modal alignment, and thus reinforces the audio-video fidelity for each other. Extensive experiments show superior results in unconditional audio-video generation, and zero-shot conditional tasks (e.g., video-to-audio). In particular, we achieve the best FVD and FAD on Landscape and AIST++ dancing datasets. Turing tests of 10k votes further demonstrate dominant preferences for our model. The code and pre-trained models can be downloaded at https://github.com/researchmm/MM-Diffusion.

Ultrasound is progressing toward becoming an affordable and versatile solution to medical imaging. With the advent of COVID-19 global pandemic, there is a need to fully automate ultrasound imaging as it requires trained operators in close proximity to patients for long period of time. In this work, we investigate the important yet seldom-studied problem of scan target localization, under the setting of lung ultrasound imaging. We propose a purely vision-based, data driven method that incorporates learning-based computer vision techniques. We combine a human pose estimation model with a specially designed regression model to predict the lung ultrasound scan targets, and deploy multiview stereo vision to enhance the consistency of 3D target localization. While related works mostly focus on phantom experiments, we collect data from 30 human subjects for testing. Our method attains an accuracy level of 15.52 (9.47) mm for probe positioning and 4.32 (3.69){\deg} for probe orientation, with a success rate above 80% under an error threshold of 25mm for all scan targets. Moreover, our approach can serve as a general solution to other types of ultrasound modalities. The code for implementation has been released.

IMPORTANCE: An interpretable machine learning model can provide faithful explanations of each prediction and yet maintain higher performance than its black box counterpart. OBJECTIVE: To design an interpretable machine learning model which accurately predicts EEG protopatterns while providing an explanation of its predictions with assistance of a specialized GUI. To map the cEEG latent features to a 2D space in order to visualize the ictal-interictal-injury continuum and gain insight into its high-dimensional structure. DESIGN, SETTING, AND PARTICIPANTS: 50,697 50-second cEEG samples from 2,711 ICU patients collected between July 2006 and March 2020 at Massachusetts General Hospital. Samples were labeled as one of 6 EEG activities by domain experts, with 124 different experts providing annotations. MAIN OUTCOMES AND MEASURES: Our neural network is interpretable because it uses case-based reasoning: it compares a new EEG reading to a set of learned prototypical EEG samples from the training dataset. Interpretability was measured with task-specific neighborhood agreement statistics. Discriminatory performance was evaluated with AUROC and AUPRC. RESULTS: The model achieves AUROCs of 0.87, 0.93, 0.96, 0.92, 0.93, 0.80 for classes Seizure, LPD, GPD, LRDA, GRDA, Other respectively. This performance is statistically significantly higher than that of the corresponding uninterpretable (black box) model with p<0.0001. Videos of the ictal-interictal-injury continuum are provided. CONCLUSION AND RELEVANCE: Our interpretable model and GUI can act as a reference for practitioners who work with cEEG patterns. We can now better understand the relationships between different types of cEEG patterns. In the future, this system may allow for targeted intervention and training in clinical settings. It could also be used for re-confirming or providing additional information for diagnostics.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

联合学习（FL）是一个新的分布式机器学习框架，可以在不收集用户的私人数据的情况下获得可靠的协作培训。但是，由于FL的频繁沟通和平均聚合策略，他们会遇到挑战统计多样性数据和大规模模型。在本文中，我们提出了一个个性化的FL框架，称为基于Tensor分解的个性化联合学习（TDPFED），在该框架中，我们设计了一种具有张力的线性层和卷积层的新颖的张力局部模型，以降低交流成本。 TDPFED使用双级损失函数来通过控制个性化模型和张力的本地模型之间的差距来使全球模型学习的个性化模型优化。此外，有效的分布式学习策略和两种不同的模型聚合策略是为拟议的TDPFED框架设计的。理论融合分析和彻底的实验表明，我们提出的TDPFED框架在降低交流成本的同时实现了最新的性能。

OD区域对之间的原点污染（OD）矩阵记录定向流数据。矩阵中复杂的时空依赖性使OD矩阵预测（ODMF）问题不仅可以棘手，而且是非平凡的。但是，大多数相关方法都是为在特定的应用程序方案中预测非常短的序列时间序列而设计的，在特定的应用程序场景中，该方法无法满足方案和预测实用应用长度的差异要求。为了解决这些问题，我们提出了一个名为Odformer的类似变压器的模型，具有两个显着特征：（i）新型的OD注意机制，该机制捕获了相同起源（目的地）之间的特殊空间依赖性，可大大提高与捕获OD区域之间空间依赖关系的2D-GCN结合后，预测交叉应用方案的模型。 （ii）一个时期的自我注意力，可以有效地预测长序列OD矩阵序列，同时适应不同情况下的周期性差异。在三个应用程序背景（即运输流量，IP骨干网络流量，人群流）中进行的慷慨实验表明，我们的方法的表现优于最新方法。

在线和离线手写的中文文本识别（HTCR）已经研究了数十年。早期方法采用了基于过度裂段的策略，但遭受低速，准确性不足和角色分割注释的高成本。最近，基于连接主义者时间分类（CTC）和注意机制的无分割方法主导了HCTR的领域。但是，人们实际上是按字符读取文本的，尤其是对于中文等意识形态图。这就提出了一个问题：无细分策略真的是HCTR的最佳解决方案吗？为了探索此问题，我们提出了一种基于细分的新方法，用于识别使用简单但有效的完全卷积网络实现的手写中文文本。提出了一种新型的弱监督学习方法，以使网络仅使用笔录注释进行训练。因此，可以避免以前基于细分的方法所需的昂贵字符分割注释。由于缺乏完全卷积网络中的上下文建模，我们提出了一种上下文正则化方法，以在培训阶段将上下文信息集成到网络中，这可以进一步改善识别性能。在四个广泛使用的基准测试中进行的广泛实验，即Casia-HWDB，Casia-Olhwdb，ICDAR2013和Scut-HCCDOC，表明我们的方法在线和离线HCTR上都显着超过了现有方法，并且表现出比CTC/ CTC/ CTC/ CTC/ CTC/速度高得多的方法。基于注意力的方法。

面部解析将像素语义标签分配为计算机的面部表示，这是许多高级面部技术的基本部分。与2D面对解析相比，3D面对解析具有更大的潜力，可以实现更好的性能和进一步的应用，但是由于3D网格数据计算，它仍然具有挑战性。最近的作品引入了3D表面分割的不同方法，而性能仍然有限。在本文中，我们提出了一种基于“ 3D-2D-3D”策略来完成3D面对解析的方法。包含空间和纹理信息的拓扑磁盘状的2D面图像通过面部参数化算法从采样的3D面数据转换，并提出了一个称为CPFNET的特定2D网络，以实现具有多种参数化脸部数据的语义分割。比例技术和功能聚合。然后，2D语义结果将成反比3D面数据，最终实现了3D面对解析。实验结果表明，CPFNET和“ 3D-2D-3D”策略都完成了高质量的3D面对解析和跑赢大于最新的2D网络以及定性和定量比较的3D方法。

影响重症患者护理的许多基本问题会带来类似的分析挑战：医生无法轻易估计处于危险的医疗状况或治疗的影响，因为医疗状况和药物的因果影响是纠缠的。他们也无法轻易进行研究：没有足够的高质量数据来进行高维观察性因果推断，并且通常无法在道德上进行RCT。但是，机械知识可获得，包括如何吸收人体药物，并且这些知识与有限数据的结合可能就足够了 - 如果我们知道如何结合它们。在这项工作中，我们提出了一个框架，用于在这些复杂条件下对重症患者的因果影响估算：随着时间的流逝，药物与观察之间的相互作用，不大的患者数据集以及可以代替缺乏数据的机械知识。我们将此框架应用于影响重症患者的极其重要的问题，即癫痫发作和大脑中其他潜在有害的电气事件的影响（称为癫痫样活动 - EA）对结局。鉴于涉及的高赌注和数据中的高噪声，可解释性对于解决此类复杂问题的故障排除至关重要。我们匹配的小组的解释性使神经科医生可以执行图表审查，以验证我们的因果分析的质量。例如，我们的工作表明，患者经历了高水平的癫痫发作般的活动（75％的EA负担），并且未经治疗的六个小时的窗口未受治疗，平均而言，这种不良后果的机会增加了16.7％。作为严重的大脑损伤，终生残疾或死亡。我们发现患有轻度但长期EA的患者（平均EA负担> = 50％）患有不良结果的风险增加了11.2％。